A Fundamental Characterization of Stability in 
Broadcast Queueing Systems 

Chan Zhou* and Gerhard Wunder* 



Abstract — Stability witli respect to a given sclieduling policy 
has become an important issue for wireless communication 
systems, but it is hard to prove in particular scenarios. In this 
paper two simple conditions for stability in broadcast channels 
are derived, which are easy to check. Heuristically, the conditions 
imply that if the queue length in the system becomes large, the 
rate allocation is always the solution of a weighted sum rate 
maximization problem. Furthermore, the change of the weight 
factors between two time slots becomes smaller and the weight 
factors of the users, whose queues are bounded while the other 
queues expand, tend to zero. Then it is shown that for any mean 
arrival rate vector inside the ergodic achievable rate region the 
system is stable in the strong sense when the given scheduling 
policy complies with the conditions. In this case the policy is 
so-called throughput-optimal. Subsequently, some results on the 
necessity of the presented conditions are provided. Finally, in 
several application examples it is shown that the results in the 
paper provide a convenient way to verify the throughput-optimal 
policies. 

I. Introduction 

In wireless communication systems input data packets arrive 
randomly at the transmitter and queue up in a buffer await- 
ing the transmission. The transmission rate for each user is 
determined by a scheduling policy which acts according to 
the system state. One major challenge for the policy design is 
to improve the system throughput by taking advantage of the 
channel variations. However, policies which consider only the 
channel state will cause unfairness among the users due to the 
randomness in the data traffic. Even more severely, some users 
may suffer from long queueing delay or even buffer overflow. 

In order to avoid this situation, the scheduling policy can 
take the queue states into account. An essential requirement 
then is that the queues are kept finite for all users and do not 
blow up over time, such that the queueing system is stable. The 
stability of the queueing system is determined by the arrival 
traffic, the transmission capacity and the applied scheduling 
policy. A scheduling policy is called throughput-optimal if 
it keeps the system stable for any set of mean arrival rates 
that lies in the ergodic achievable rate region. Throughput- 
optimality is a desirable feature of scheduling policies, since 
the system can offer a maximal possible traffic load and keeps 
all queues stable at the same time. 
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There already exist a number of throughput-optimal 
scheduling policies in previous work, e.g. the Maximum 
Weight Matching (MWM) policy [l]-[4], the exponential rule 
[5], the Queue Proportional Scheduling (QPS) [6], the Idle 
State Prediction Scheduling (ISPS) [7]. Throughput-optimality 
of these policies is proven using techniques which are adapted 
to the particular policy and in fact the proofs can be rather 
involved [1], [5]-[7]. Eryilmaz et al. provided some sufficient 
conditions under which the scheduling policies are throughput- 
optimal [8]. However, these conditions are quite restricted and 
do not include all throughput-optimal scheduling policies (e.g. 
exponential rule, QPS, ISPS). 

In this paper, we consider scheduling policies in a broadcast 
system and give general conditions for their throughput- 
optimality. The scheduling policy is formulated as the solu- 
tion of a weighted sum rate maximization problem differing 
only in the choice of the weight factors. Then we show 
that throughput-optimality can be verified solely by checking 
characteristics of the weight factors. It is shown that the weight 
factors of a throughput-optimal scheduling policy only need 
to satisfy two conditions: 

1) If the total queue length in the system becomes large, 
the change of the weight factors between two time slots 
tends to zero. 

2) If the total queue length in the system becomes large, the 
weight factors of "nonurgent" users, whose queues are 
bounded while the other queues expand, tend to zero. 

The proofs are derived by using some special Lyapunov 
functions in connection with theorems in differential geometry. 
It is proven that the presented conditions are also necessary 
and indeed cover all queue-length based throughput-optimal 
scheduling policies. We apply these results to well-known 
scheduling policies and show their throughput-optimality. 

The remainder of this paper is organized as follows: Section 
nil describes the system model including some assumptions on 
the rate and arrival processes. In this section we also introduce 
the definition of stability and throughput-optimality. General 
sufficient conditions for throughput-optimality are presented 
in Section |III] The necessity of these conditions is proven in 
Section HV] Some applications of our results are shown in 
Section W\ Finally we conclude in Section [VTl 

Notations: We use boldface letters to denote vectors and 
common letters with subscript are the elements. ||x||i denotes 
the /i-norm of the vector x and ||x|| is a arbitrary norm of 
X. E{x} denotes the expected value of random variable x. 
Furthermore we use A'^ to denote the complement of a set A. 



The probability function is denoted as Pr{-}. The indicator 
I{ } equals 1 if the argument is true and equals otherwise. 

II. System Model 

A. Physical layer 

We consider a single cell downlink system in which a base 
station simultaneously supplies M mobile users. The channel 
between the base station and each user is assumed to be 
constant within a time slot and varies from one time slot 
to another in a stationary i.i.d. manner. The channel state 
of user i in the 71-th time slot is denoted as hi{n) G S, 
where S is an arbitrary countable or uncountable set, and 
all channel states of the user set A4 := {!,..., A/} are 
collected in the vector h(n) e S^^ . Here the set S is used 
to indicate that the general approach is not restricted to a 
specific transmission scheme. For example in a MIMO system 
the channel state can be described as a matrix of complex 
channel gains such that hi{n) G C"''"' where rir, rii are the 
number of transmit and receive antennas at the base station 
and mobiles, respectively. Likewise for an OFDM system the 
channel state can be defined as a vector of complex channel 
gains on each subcarrier hi{n) G where K is the number 
of subcarriers. Furthermore we assume that the channel state 
information is, either perfectly or partially, accessible at the 
receiver and the transmitter. In the n-th time slot the data is 
transmitted through the channel at the rate r(ri) G 1$.^ with 
the transmission power p(n) G . 

We define a resource allocation policy V with rate alloca- 
tion r^(n) and power allocation Tp^{n). The allocated rate 
satisfies r^(n) G C (h(n), p''^(n)), where C (h(7i), p(n)) is 
the achievable rate region of the broadcast system with the 
channel state h(n) and the transmission power p(n). We don't 
specify the achievable rate region C (h(7i), p(ri)) so that the 
results in the paper can be applied to different transmission 
schemes. 

The entire achievable rate region of a system with maximum 
power constraint P in time slot n is given by 



C(h(n),p) := U C(h(7i),p^(n)) 



(1) 



veil 



where VI is the set of feasible policies 



V 



In general, the rate region C \^{n), Pj might be non-convex; 
then we consider the convex hull of the achievable rate region. 
Any point on the convex hull of the region C{h.{n),P) is a 
solution of the maximization problem 



r(/x,h(7i)) 



argmax fi r, 



(2) 



feC(h(n),P) 



where G Rl' is the set of weight factors. It is important 
to stress that even though the weight factors are fixed and 
independent of the channel state h{n), the rate allocation 
r(/i,,h(n)) depends on the channel state due to the maxi- 
mization problem over C(h(7T,), P) in (|2|i. Observe that weight 



factors fi also represent a normal vector of a supporting 
hyperplane which is tangential to the convex hull at the point 
r(/2,h(n)). 

Then, the ergodic achievable rate region is defined as 



C{P) 

^ n 

IImII=i 



n, ■ 



,rAi : H r < 



max fi^r 

feC(h(n),P) 



(3) 



Note that the definition ^ coincides with the capacity region 
given in [9], [10], when C {h.{n), p{n)) is the capacity region 
of parallel degraded channels. Furthermore, it is shown in [3], 
[4], [11] that the region is also the maximal achievable stability 
region. In general the characterization of the achievable rate 
region C(h(7i),P) and the solution of the weighted sum rate 
maximization problem is complicated. Considering practical 
constraints such as finite code and modulation scheme and 
imperfect channel state information, the ergodic achievable 
rate region C{P) can be much smaller than the information- 
theoretical capacity region. Thereby the instantaneous achiev- 
able rate region C{h{n),P) might be non-convex or even be 
restricted to a set of some discrete rate points. The results in 
this paper can also be applied to these practical systems as 
long as a solution of the problem in (|2]i exists (i.e. for an 
discrete OFDMA system described in [12]). 

It is easy to show that the ergodic achievable region given 
in (O is convex. According to the convexity, any boundary 
point of the region C{P) is a solution of the problem 



argmax/x r. 



(4) 



fec(p) 



Likewise here the weight factors fi also represents the normal 
vector of the boundary at the point (fi) (see Fig[T]). 




Fig. 1. The solution of tlie weight maximization problem j4) is a point 
on the boundary of the ergodic achievable rate region C{P). The vector of 
weight factors fj, can be interpreted as the normal vector of the boundary at 
the obtained point 



B. Medium Access Control (MAC) layer 

Assuming that the transmission is time-slotted, data packets 
arrive randomly at the MAC and queue up in a buffer reserved 
for each user i <E M. Simultaneously the data is read out 
from the buffers according to the system state, i.e., the random 
channel state and the current queue lengths. Thus, the system 
can be modeled as a queueing system with random processes 
reflecting the arrival and the departure of data packets. 

Denoting the queue state of the i-th buffer in time slot n €N 
by Qi (ri) and arranging all queue states in the vector q(n) G 
M.^ , the evolution of the queueing system can be written as 

q(7i + l) = [q(?7.)-r(n) + a(n)]+, (5) 

where [x]f — max{0, Xj}, Mi G AA. Vector a(n) G MJ^ is a 
random vector denoting the amount of arrival packets during 
the n-th time slot and vector r [n) G is the amount of 
transmitted data. 

Without loss of generality we set the length of a time slot 
r = 1 so that a (n) and r (n) are equal to the arrival and 
transmission rate during the time slot n. We assume that the 
size of a data packet is constant. To simplify the notation we 
set the packet size to 1 unit without loss of generality. 

Eqn.© can also be formulated as 

q (n + 1) = q (n) + a (n) - r (n) + z (n) , (6) 

with 

^ , , f qi{n) + a,{n) ~ri{n)>Q 

' y ri{n) — qi{n) — ai{n) otherwise 

We assume that the sequence of arrival bits forms an i.i.d. 
sequence of variables over time. For technical reasons we 
assume that the arrival bits ai{n) are uniformly bounded by 
some real constant Ca > 0. 

The transmission rate r (n) is determined by the applied 
scheduling policy. We consider scheduling policies which are 
independent of the time index and define the policies as the 
mapping from the cartesian product of the set of channel gains 
h (n) and queue lengths q (n) to the set of transmission rates. 
The rate allocated by policy V is denoted as r''(h(?7), q(n)). 
Further we make the technical assumption that the maximum 
transmission rate r^(h(n), q(n)) are uniformly bounded by 
some real constant Cr > 0. Under these assumptions, the 
considered queueing system can be modeled as a ?/;-irreducible 
Markov chain with 7/)-irreducible measure Sq where Sq denotes 
a Dirac measure at zero [13]. 

C. Stability 

The stability of an ^/^-irreducible Markov chain can be 
defined in different manners. We first introduce the definitions 
of recurrent and transient Markov chain as given in [13]. 
These definitions are based on the measure of the occupation 
time 

oo 

7JA := ^I(q(n) G A) 

n=l 



which gives the number of visits in a set ^ G M^p by a Markov 
chain after time zero. 

Definition 1: A Markov chain is recurrent, if it holds 
E {tja} = +00, Mx e A for any set A G Additionally, if 
the Markov chain admits an invariant probability measure tt, 
then it is positive recurrent. 

If the Markov chain is positive recurrent, it is also weakly 
stable [14] so that it holds 

lim Pr(||q(n)|| > B) < e 

n — »+oc 

for any e > and some constant B > 0. 

Definition 2: A Markov chain is transient, if there is a 
countable cover of M.^ with uniformly transient sets, i.e. there 
is a constant C with E {77^} < C, Va; G yl. 

In this paper we also apply another stability definition as it 
is used in [8]: 

Definition 3: A Markov chain is called f-stable, if there 
is an unbounded function / : M.^ E_|- so that for any 
< i? < +00 the set S {x : / (x) < B} is compact, and 
furthermore it holds 

limsupE{/(q(n))} < +00. (7) 

n — *+oo 

The function / is unbounded in all positive directions so 
that /(q(7i)) goes to infinity when ||q|| goes to infinity. 
Choosing directly / (q) = ||q||. Definition [3] is equivalent 
to the definition of strongly stable [14]. Moreover, it is easy 
to show that for any / (q) which grows faster than ||q|j, 
inequality d?) implies that the Markov chain is strongly stable. 

Denoting the mean of the arrival bits ai{n) per time slot 
as Pi collected in the vector p G , we call a vector of 
arrival rates p stabilizable under V when the corresponding 
queueing system driven by some specific scheduling policy V 
is positive recurrent. 

It is well-known that any vector of arrival rates inside the 
ergodic achievable rate region C{P) is stabilizable (e.g. under 
MWM policy) and any vector of arrival rate outside C{P) 
is not stabilizable [8], [11]. Thus a scheduling pohcy is now 
called throughput-optimal if it keeps the Markov chain positive 
recurrent for any vector of arrival rates p G int(C(P)), where 
int(C(i-')) denotes the interior of the ergodic achievable rate 
region C{P). 

III. Stability Conditions 

We consider scheduling policies which solve the weighted 
sum rate maximization problem 

r'^(h(n),q(n)) = argmax /x^ (q(n)) r, (8) 
fec(h(n),p) 

where fi^ (q) denotes the weight vector for some queue state 
q determined by a scheduling policy P. Note that the weight 
factors fj,^ (q) depends solely on the queue state. In Section 
HVl we argue for the necessity of this assumption. It is worth 
noting that the solution of the optimization problem in ^ is 
a boundary point of the convex hull of C{h.{n), P). However, 
such a rate allocation is possibly not uniquely defined by the 
weight vector pi^{q) (which is often the case). Nevertheless 



we can enforce uniqueness by invoking e.g. additional con- 
straints on the allocated rate vector which, by the way, do not 
affect the line of proof in Theorem [T] 

One of the well-known throughput-optimal scheduling poU- 
cies is the MWM policy, which uses the weight vector 



and hence 11 /^■''(q) I 



1. Since the magnitude of the 



(9) 



The throughput-optimality of the MWM policy in general 
multiple-access and broadcast channels is proven in [3], [4]. 
Despite its simple form the MWM policy has satisfactory 
delay and fairness properties and is applied in several systems 
including MIMO [15] and OFDM. 

A further class of throughput-optimal scheduling policies 
use the exponential rule [5]. Here, the weight vector is given 
by 



fJ-t (q) === l^exp 



(10) 



where 7i,...,7j\f, Q!i,...,Q!a/ are arbitrary sets of positive 
constants and the positive constants (3 and rj G (0, 1) are fixed. 

A more generalized class of throughput-optimal scheduling 
policies is presented by Eryilmaz et al. in [8]. The weight 
factor iii{qi) is given by a function of qi satisfying the 
following conditions: 

1) Hi{qi) is a nondecreasing, continuous function with 
lim,,_^+oo yUj(ft) = +00. 

2) Given any Ci > 0, C2 > and < e < 1 there exists 
a B < +00, such that for all qi > B and Vi e M, 

(1 - e)^J-iiqi) < - Ci) 

< ^^^iq^ + C2) <{l + e)^l^iq^). (11) 

Condition ( fTTI ) implies that the relative difference 
A'.(9.±C)-M.(gi) (g[j(js toward zero for constant C if is 
large. Hence the scheduling policies using weight functions 
such as = e'^' do not belong to this class. Actually, it can 
be proven that these scheduling policies are not throughput- 
optimal. 

The conditions given in [8] cover quite a large class of 
throughput-optimal scheduling policies. However, the weight 
factor /i,;(gi) is exclusively calculated by qi and independent 
of the queue length of other users, which is a rather strict 
constraint. In general the weight factor is determined by the 
queue state of all users, i.e. 



fJ-i 



Some examples are the aforementioned policies using expo- 
nential rule, the QPS and the ISPS. Unfortunately the results 
in [8] can not be applied in these cases. 

Here, we give generalized sufficient conditions for 
throughput-optimality. The conditions are presented by charac- 
terizing the corresponding weight vector fi^ of the scheduling 
policies. In the following we consider the normahzed weight 
vector 

M^(q) 



lA*^(q) 



(12) 



weight vector does not affect the solution of the maximization 
problem (|2|i, namely the scheduling decision, we only need to 
consider the direction of the vector. Thus the normalization of 
weight factors fJ-'^^iq) can be done without lose of generality. 

Theorem 1: Any vector of arrival rates p G int(C(P)) is 
stabilizable under the scheduling policy V, if its corresponding 
normalized weight vector /i(q) given in Eqn. (fT2l i fulfills the 
following conditions: 

1) Given any < ei < 1 and Ci > 0, there is some 
i?i > so that for any Aq eM^^ with |[Aq|| <Ci, we 
have \ j2i (q + Aq) — fii (q) | < ei for any q G M.^ with 
||q|| > Bi, yi e M. 

2) Given any < £2 < 1 and C2 > 0, there is some 
i?2 > so that for any q G with ||q|| > B2 and 
qi < C2, we have /ii(q) < £2, for any i G M.. 

Moreover, for any arrival process with p G int(C(P)), the 
queueing system is f-stable under the given policy V, where 
/ is an unbounded function as defined in Definition |3] The 
exact formulation of / depends on the weight function /i(q). 

In order to simplify the notation, provided the limit exists, 
we can also write the two conditions as 



lim l/i. 

||q|H+oo 

lim 

||q|H+oo 



q + Aq) - (q)| 

P-i (q)l = 0, 



0, 



lAqll <Ci 
qi < C'2, 



where 



|q|| +00 is any path in M.^ with unbounded norm. 
The conditions are interpreted in the next section and their 
necessity is proven in Theorem [5] Before we give a proof 
of Theorem [T] we compare the conditions in Theorem [T] and 
the conditions given by Eryilmaz et al. To this end, let fii{qi) 
be the function dependent only on qi and satisfies Eryilmaz's 
conditions. We consider two different cases: qi < +00 and 
qi +00. If qi is bounded and ||q|| goes to infinity, then we 
have some j ^ i with qj +00. According to Eryilmaz's 
conditions, it follows that iii{qi) < +00 and Hj{qj) +00. 
We normalize the weight vector so that we have 



lim ^i(q) 

||q|H+oo 

lim 



= 



and Condition 2) in Theorem [T] is fulfilled. It holds also that 
lim|[q|[_,^oo /ii(q + Aq) = as long as ||Aq|| is bounded. 
Thus lim||q||^^oo |/2i(q + Aq) — fii{q)\ = and Condition 
I) is satisfied. 

If qi +00 we only need to check the first condition in 
Theorem [T] Suppose |jAq|j < C for some constant C > 0, 
we have Aqi < C, Vz G A^. After the normalization we have 

/2,(q + Aq) = V...^A..^ ^ (13) 



1 + E 



jT^i tJ.i{qi+Aqi) 



As ||q|| +00, if there are other users j ^ i with qj 
then according to (fTTI) we have 



-CG, 



lim 



fii{q.i + Aqt) ||q|H+. 



Otherwise if qj is bounded, it holds 

lim M.-fe- + Ag.-)^o^ 
||q|H+oo Hi(qi + Aqi) 

Considering the both situations and substitute them in (fTsT l. 
we have lim||q||^+oo |/ii(q + Aq) - ^i(q)| = and the 
condition is satisfied. Hence we conclude that the class of 
policies in [8] is indeed included in the theorem. 

Proof: The proof is given in Appendix A. ■ 

So far we considered scheduling policies based on the cur- 
rent queue state. In some situations the queue state information 
might be imprecise or delayed. These cases occur even more 
frequently in an uplink system, where the queue state infor- 
mation has to be quantized and transmitted from the mobile 
terminal to the base station through a signaling channel. In this 
paper we mainly consider the downlink system, however we 
emphasize that the physical layer described in Section can 
be generalized as the achievable rate region is independent of 
the transmission schemes. Thus our results can also be applied 
to multiple-access channels in uplink systems if we replace the 
term downlink achievable rate region with uplink achievable 
rate region in our system model. 

Theorem [T] has an interesting interpretation: suppose the 
scheduling policy determines the rate allocation based on the 
quantized queue state information q(n) with some quantiza- 
tion error e{q{n)), then we have ci{n) = q(7i)+£(q(n)). If the 
weight vector /i(q(n)) determined by the scheduling policy 
satisfies the conditions given in Theorem [T] when ||q(n)|| is 
sufficiently large, it is easy to show that 

|^,;(q + Aq+£(q + Aq)) - /2j (q+£(q))| < ei 

for any < ei < 1 and bounded Aq, and if qi is bounded, it 
holds 

/2j(q+£ (q)) < £2 

for any < £2 < 1 as ||q(7T.)|| is sufficiently large. Thus the 
scheduling policy is also throughput-optimal. 

If the obtained queue state information has A71 time slots 
delay, we have q(7i) = q(ri — An). Since the transmission and 
arrival rate are bounded, we have q(7i) = q(n) + where 
the error Sd caused by delay is also bounded. Similarly it can 
be shown that the stability conditions can also be applied in 
this case. 

IV. Necessity of Stability Conditions 

In the previous section we presented sufficient conditions 
for throughput-optimal scheduling policies. To do so, the 
scheduling problem is formulated as a weighted sum rate 
maximization problem, and, hence, the rate allocation of these 
scheduling policies always lies on the convex hull of the 
instantaneous achievable rate region C(h(7T,), P). Additionally, 
the weight factors fi^ are independent of the instantaneous 
channel state. Actually, we can observe that all existing 
throughput-optimal policies have these general characters (al- 
though it is not explicitly noticed in previous works). In fact it 
is an inherent necessity of any throughput-optimal scheduling 
policy which is expressed as the following theorem. 



Theorem 2: If the queue lengths are sufficiently large, 

1) a throughput-optimal policy always allocates the rate 
vector on the convex hull of the instantaneous rate region 
thus the rate allocation can be formulated as a weighted 
sum rate maximization problem. 

2) Furthermore, the weight vector fi^ in the maximization 
problem is independent of the current fading state h(n). 

The theorem is proven in [7]. Next we consider the necessity 
of the stability conditions given in Theorem [T] It is immedi- 
ately clear that the conditions are not universally necessary. 
In some specific scenario the achievable rate region has no 
unique supporting hyperplane for some point on the boundary. 
Hence, the weight vector on these points is not unique. A 
typical example is a rate region with only two available rate 
points on the boundary. In this case a throughput-optimal 
scheduling policy can be characterized by a weight function 
where the image consists of two points in only. Obviously 
this weight function does not satisfy Condition 1) in Theorem 
[U To fix this problem, we argue that in the wireless case 
the achievable rate regions are varying over time and further 
the policies must be defined for any possible configuration 
of C{h.{n), P). Hence, to prove the necessity we will only 
considered those achievable rate regions where the weight 
vector is unique for every boundary point. 

Before we prove the necessity under above assumptions, we 
want to give some intuition about why these conditions must 
hold in general. Recall that a throughput-optimal policy should 
keep the queues stable for any mean arrival rate p inside the 
ergodic rate region C{P); so if the arrival rate vector p lies 
close to some boundary point r* of C{P) corresponding to a 
weight vector p*, heuristically, the weight vector determined 
by a throughput-optimal scheduling p,^ should be in the close 
neighborhood of p* for almost all time slots. Condition 1) in 
Theorem [T] ensures now that the weight vector varies smoothly 
between two time slots if the queue lengths become large. Thus 
for the above situation, it guarantees that the weight vector p,^ 
does not leave the neighborhood of p* in almost all time slots. 
Condition 2) in Theorem[T|guarantees that no rate is wasted on 
"nonurgent" users. If the queues of some users are bounded 
while the other queues expand, the scheduler should reduce 
the weights on these users and save these rate resources for 
other users. 

Theorem 3: A scheduling policy V is not throughput- 
optimal, namely there exists some arrival process with p G 
int(C(P)) which is not stabilizable under the policy V, if the 
policy has one of the following characteristics: 

1 ) The change of the weight vector between two time slots 
is not negligible, i.e., there is some constant < 7 < 1 
and e > so that it holds 

1 ^ 

„i™ ^^i{llA(q(" + i))-M(qW)ll >e}>7 

n—1 

(14) 

for any q(7i), n G N, with probability 1. 



2) There is some user i £ A4, whose weight factor is not 
negligible, i.e., there is some constant < 7 < 1 and 
e > so that it holds 

1 ^ 



n=l 



for any q(n), n G N, with probability 1. 
Proof: The proof is given in Appendix B. ■ 
Applying the results in Theorem [3] it can be proven that the 
policies using certain exponential functions as weight factors 
are not throughput-optimal. The details are given in Section 

m 

V. Applications 

In this section we prove the throughput-optimaUty of some 
well-known scheduling policies. Note that the throughput- 
optimality of these policies are already proven in the previous 
works, here we use our results to perform the proof in a 
different way and show the applicability of our results. In this 
section we also use the necessary conditions in Theorem [3] 
and verify some policies which are not throughput-optimal. 

Normalizing the weight factors given in (|9]l, it is easy to 
show that the MWM policy satisfies the conditions in Section 
Hin and, hence, it is throughput-optimal. In the following we 
use the results to check the throughput-optimality of several 
other scheduling policies. 

A. Exponential Rule 

NormaUzing the weight factors given in ( fTol i. we have 



7't 



It 



( o-jqj-aiqi 

V/3+5(q)||q||^' 



(16) 



Recall that 71,..., 7m, oi, q;j\/, /3, 77 are predefined 
constants in the scheduler and we define a (q) := 
( J_ . "^q 



M ||a||2||q||„ 

In order to check the first condition in Theorem [1] we need 
to show that 



lim (q + Aq) - ^H (q) 

||q||->+oo 

If ||Aq|| is bounded, it holds lim||q||^+, 



limn 



, a (q) and 



lim 



= 0. (17) 
,5(q + Aq) = 



||q|H+oo /3 + a(q+ Aq) ||q + Aq||^ 



= lim 



ajqj - UiQi 



||qP+oo /? + a (q) ||q||^ 

for all i,jGA4 so that the Eqn. ( fTTI i follows. 

Considering the second condition in Theorem [H if qi is 
bounded as j|qj| increases, there is another user j who has the 
longest queue so that qj > then we have 

lini ^ li^^ 

H\\^+oo P + a{q) \\q\\'^ l|q|H+oo a (q) ||q||^ 

(18) 



since 77 < 1. 

Substituting ( fTSl l in Eqn.(fT6ll, it follows 



lim fl^{q) 

||q|H+oo 







which fulfills the Condition 2) in Theorem [T] and the 
throughput-optimality is proven. 

B. QPS 

QPS from [6] is a scheduling policy which has good delay 
and fairness performance in the downlink. Applying QPS in a 
broadcast system with random arrivals, each user's queueing 
delay becomes equal as n ^ +cxd. Additionally, if the queue 
state is initialized by q (0) > and there is no new packet 
arrivals after n ~ Q, which can be considered as a draining 
problem, the QPS minimizes the expected draining time until 
all the buffers are cleared. 

The rate vector is allocated so that 



E{r^ (h(n),q(n))} 



q (n) max x, 

a:q"eC(P) 



where a; is a scalar. According to the policy, the weight 
vector is chosen as the norm at the boundary point of C{P) 
where E | (h, q)| q} is proportional to q. Fig|2] shows the 
expected rate vector allocated by QPS compared to MWM 
policy and the ergodic achievable rate region in a 2-user 
scenario. 



boiindLU-y of C(P) 




MWM 



Fig. 2. The weight vector of QPS and MWM policy. For MWM ^j.'^'WM ^ 
q and for QPS the weight vector fjft^^ is the norm at the boundary point 
of C ( P ) where E { (h, q) | q} is proportional to q 



The weight vector in QPS is found by Geometric Program- 
ming. Unfortunately, there exists no explicit function of the 
weight factor /x and the rate allocation in QPS. Therefore 
the verification of its throughput-optimality is not easy using 
standard methods. The presented general approach in this 
paper is particularly suitable in this case. 



Since the weight vector is determined by the normalized 
queue state for bounded ||Aq|| we have 

lim jli{q + Aq) 

||q|H+oo 



= lim 

||q|H+oo 



q + Aq 



llq 



lim fJ-i{q). 



If (7i < +CXD we have 



and hence 



lim 

llqlH+= 



0. 



Both conditions in Theorem [T] are satisfied and the QPS 
policy is throughput-optimal. 

C. ISPS 

Another scheduling policy which also provides superior 
delay performance is the ISPS in [7]. If there is no new packet 
arrivals in the system and the queue state is initialized by 
q° > 0, the policy minimizes the average queueing delay 



D{N) 



7\/ AI N 



MN ^ E{a, (n)}' 
i=i „=i V i\ jy 



(19) 



where N is the length of the observed time slots. 

The weight factor /ii(q) is determined by the estimated 
service time r]i such that 



Vi (q) 



(20) 



where ai,aj G M+ are some predefined constants. Parameter 
?7i(q) is the expected service time of user i if the ISPS is 
applied in the system and no new packet arrival occurs. Then 
in the time slot n > rji{q) the buffer of user i is completely 
emptied and the corresponding transmitter is in idle state. 
The estimated service time rj is obtained with an iterative 
algorithm. Hence there is also no explicit function of i-ii (q) in 
ISPS. 

However, we can see that if || Aq|| < +oo, it follows Arji := 
|7/i(q + Aq) -?7i(q)| < +oo. If +oo, then ?/i(q) 

+00. It holds 

lim 1/2, (q + Aq) - p,i (q)| 

|q||^ + oo 



lim 

llqlH+oo 



Vj (q) 



If Qi is bounded we have 77i(q) < +oo and 



lim > 

||q|H+oo^ 



and 



^i(q) 



lim ii,{q) 

||q||-» + oo 



lim 



'?j(q) 



0. 



||q|H + oo ''■_(q) 

at 

Thus the throughput-optimality is proven. 
D. Exponential Functions as Weight factors 

Applying Theorem [3] it can easily be shown that the policy 
using exponential weight function such as /.t^ (q) = e'^' is not 
throughput-optimal. We consider a point r* on the boundary 
of ergodic achievable rate region C{P) and its corresponding 
normal vector /x* with ^* > 0, V« S A^. Suppose the expected 
arrival rate vector p lies close to the boundary point r*, in 
order to keep the system stable, the weight factors jl should 
be frequently chosen close to /x* so that there is some constant 
7i > with 

A*— *+oo iV — 



n=l 



for arbitrary > and any q(n), n G N, with probability 1. 

If \p,i (q(n)) — fJ,*\ < 9, according to the definition of fu (q) 
we have 

/2, (q(n)) e« 



< 



(21) 



for some constant Ci , C2 > 0. We denote by 

Aq := qin + 1) - q(n) = a(n) - r''^ {h{n) , q{n)) . 

Due to the randomness of a{n) and h(7i), for a particular user 
i G M, there is some probability 72 > that 

Pr {C3 < Aq, < C4, Aq, < 0, Vj ^ i} > (22) 

for some constants C3, C4 > 0. Then it holds 

p., {q{n + 1)) - jli {q{n)) 

qi+Aqi qi 



>- 



>- 



E 



ei3 



According to d^lT i we have 

/ii (q(n + 1)) - /2, (q(7i)) 

6^3-1 



for some constant e > 0. Then, combining with ( |22] | and our 
i.i.d. assumption the inequality ( fT4l ) in Theorem [3] holds with 

1 ^ 

— f + CX3 iV ^ ' 



with probability 1 and the queueing system is not stable. 



VI. Conclusions 

We have presented sufficient and necessary conditions for 
throughput-optimality of queue-length based scheduHng poH- 
cies. For a wide class of arrival and channel models these 
conditions guarantee that if the mean arrival rate lies inside 
the ergodic achievable rate region, the system is stable. With 
application examples such as Queue Proportional Scheduling 
and Idle State Prediction Scheduling we have shown that the 
stability can even been proven in cases where conventional 
proof techniques fail. 

VII. Appendix 

A. Proof of Theorem Q] 

Stability can be proven by checking the so-called Lyapunov 
drift criteria as given in [1], [3], [14]. That is to say if we can 
find some non-negative T^(q) : MJ^ R+, some 6* > and 
a compact region ;B := {q : ||q|| < B} such that 



E{F(q(n + l))|q(n)} < +oo 
Ay (q(n)) < -d 



Vq (n) e B 

(23) 

Vq(n) ^S, 

(24) 



the queueing system is positive recurrent. Here, Al/(q(n)) 
is the one-step drift defined as 

Ay (q (n)) := E { y (q (n + 1)) - ^ (q (n))| q (n)} . 

Furthermore, if for some 6* > 0, it satisfies 



Ay (q) < -Of (q) , 



V||q|| >S 



(25) 



for some B > Q and unbounded positive function / (q), it can 
be shown that the queueing system is f-stable. 

We carry out the proof in two steps. First, we prove the 
throughput-optimality for those policies, whose weight factors 
/i(q) fulfill the integrability condition in Eqn.(|26l). The weight 
factors in those policies can be regarded as the normalized 
gradient of a certain potential field V{q). We show that the 
expected drift Ay (q) satisfies the inequality dZST l and hence 
the system driven by those policies is stable. In the second 
step, we extend the results to all other policies whose weight 
factors are not integrable. It is shown that if the policies 
fulfill the condition given in the theorem, their weight factors 
/i(q) can be approximated by some functions /i(q) which are 
integrable. Then we prove the drift condition Ay (q) for those 
policies and establish the stability. 

Firstly, we analyze the subclass of weight functions whose 
/2i(q) are continuously differentiable. Furthermore, we assume 
that the weight functions satisfy the integrability condition, 
i.e., 

difir (q)) _ dip. J (q)) 



dqj 



dqt 



(26) 



For scheduling policies with this kind of weight functions, we 
have the following lemma. 



Lemma 1: If Eqn. ( |26] | holds for all q S , then any 
vector of arrival rates p E int(C(P)) is stabilizable under the 



corresponding scheduling policy as long as /i(q) fulfills the 
conditions given in Theorem [T] 

Proof: Condition ( |26] | implies that the vector field defined 
by /i(q) has the path independence property, namely the 
integral of /i(q) along a path depends only on the start and end 
points of that path, not the particular route taken. According to 
Poincare lemma, the vector field /i(q) is completely integrable 
and it is the gradient of a scalar field, that is to say, there exist 
some function /(q) : — > R+ with 

5/(q) 



(27) 



Setting the value of /(q) at the origin equal zero, /(q) at the 
point q can be calculated by 



/(q) 



H{tq) qdt, 



(28) 



where q irAr- is the normalized vector of q. Since each 
^ _||q||2 ^ 

element of /x(q) is larger than or equal to zero, /(q) is a 
positive function. Moreover, if ||q|| becomes large, according 
to Condition 2) in the TheoremlT] for i-th queue with bounded 
qi, qi ^ results in /ii(q) 0. Then for other queues with 
j2j{q) > Cfj^, qj grows with increasing ||q|| and we have qj > 
Cq for some Cq and > 0. Thus it holds 

A(q)^q > C 

for some C > if ||q|| is sufficiently large. Considering 
Eqn.(l28]), it follows that /(q) +oo as ||q|| — > +oo. 
Therefore, /(q) is a positive, unbounded function as we used 
in Definition [3] 

Observing a new vector field defined by i'(q) = ./(q)/i(q)' 
we have 

d{uiiq)) 9(/(q)/iz(q)) 



dqj 



dqj 

P-j (q) fii (q) + 
Mj (q) f^i (q) + 

a(/(q)/2,-(q)) 
dqi 

5(^,(q)) 



dfij (q) 

dqi 
dfM (q) 



G M. 



(29) 



pM 



Condition ( [29] l ensures that i>'(q) is also the gradient of a scalar 
field and there is a function y(q) 

dV{q) 



with 



/(q)M*(q), 



where /(q) is the magnitude of the gradient and /i(q) is the 
direction of the gradient. Set V{0) = and y(q) at the point 
q is 



y(q) = 



|q||2 



/(iq)M(iq) tidt. 



It is easy to shown that the function y(q) is also a positive, 
unbounded function. We use the function y(q) as our Lya- 
punov function in the proof. 



The first condition of the Lyapunov function given in ( |23] ) 
is satisfied as long as the arrival rates ai{n) and transmission 
rates ri{n) are bounded. Next we analyze the second condi- 
tion, namely the drift of V{q) of the queueing system. For 
convenience we use the superscript to denote the time slot in 
the following. 

Using the mean value theorem of differential calculus we 
have for some q" between q" and q""''^ i.e. = cti?" + 
(1 - ai) +\ e M, for some e [0, 1] 



Ay(q") 



=En^/(q")/2.(q")(ar-rr) 

I i=l 
( M 

+ E<^ ^/(q")M,(q")^ 
I 1=1 

Considering the first part in dSTl ). we have 
E|E/(q")M.(q")«-^") q"| 

(MM \ 
^/2,(q")p.-^/i.(q")E{rr|q"} 
i=l i=l / 

( M 

+ E ^ |/(q")M.(q") - /(q")Al.(q")ll< - 



(30) 



(31) 



(32) 



1=1 



(33) 



Since 



E{r"|q"} = arg max /i(q")^r- 

?ec(p) 



for any p G int(C [P]), we can always find some F > 0, so 
that 



E^/i,(q)(p,-rr) 
Hence the first part in ( |33] | 



< -r. 



M 



M 



/(q") - ^A2,(q")E{rr|q"} 

\i=l 1=1 / 

< - r/(q"). 

For the second part in ( [33] ), we define Aq = q" — q". Then 

/(q" + Aq) - /(q") 
= / /i(q" + fAq)Aqdt 

< r||Aq||idi=||Aq||i 
Jo 

Since a" and r" are bounded, we choose some C3 > 1 so 
that < C3 and rf < C3 for all i. Then |jAq||^ is bounded 
by 2A/C3 and we have 

l/(q")-/(q")l <e3/(q) 



for any given £3 > and sufficiently large ||q||. According to 
Condition 1) in Theorem [l] we also have 

|/2,(q")-/2,(q")| <6i. 

Then if |lq"|| is sufficiently large, 

f 

E l/(q")/^«(q") - /(q")/^^(q")ll< - ^ 

I i=l 
f 

<2C3E ^ (/(q") + e3/(q")) ) + 



i=l 



2C3E^/(q")/2,(q") 



=(2MC3ei + 2^363 + 2AfC3eie3)/(q") 



(34) 



holds for any ei, 63 > 0. Hence we have ai — * when 

l|q|| ^ +00- 

Now we consider the second part in dJTT i. 



A/ 



I 1=1 

<E^/(q")A2,(q")zr 



(35) 



1=1 



E 5]|/(q")/2.(q")-/(q")/2,(q" 



(36) 



For the first part in ( [36l ), since z" < is bounded by 
the current rate region, E { z" | q" } is bounded by the ergodic 
achievable rate region so that for some C4 > we have 



(37) 



We define the set Q := {i : zi > 0, ?' e M). Since rf < C3 is 
bounded by C3, then < C3, Vz G If ||q"|| is sufficiendy 
large so that ||q"|| > A/C3, we can exclude the case Q = M. 
According to Condition 2) we have /2i(q") < €2, E G for 
arbitrarily small £2. Then 



E ^/(q")A2,(q")zr 



q" \ < MC4e2/(q") (38) 



holds. 

Using the same proof method as for ( l34l i it can be shown 
that the second part in ( |36] | can be bounded by 0-2 /(q") for 
any (T2 > 0. 

Define 6* = F — (Ti — MC4e2 — (J2 and choose ui, 02, £2 
so that 6* > we have the drift 



Ay(q") < -0/(q") 



(39) 



and which is negative and the Markov chain is positive 
recurrent. ■ 
Lemma [T] is applied to weight functions which are com- 
pletely integrable. In general the weight functions don't have 
to meet the integrability condition ( |26] | or can be even not 



continuously differentiable. However, it can be shown that 
if the weight function /i(q) has the properties described in 
Theorem[ri it can be approximated by some (at least piecewise 
integrable) function /i(q). The following lemma help us to 
achieve our main result. 

Lemma 2: If the function /i(q) fulfills the Condition 1), 2) 
in Theorem[T] then there exists a positive, unbounded function 
/ : R+ as given in Definition [3] and a positive, 

continuous, piecewise differentiable function V : R*^ — > M+, 
such that it holds 



dqi 



/(q)/x,(q),V^ £ M 



(40) 



on each differentiable subdomain of V, and 



lA»(q) - Ai*(q)l < e4,Vi e M, 



(41) 



for any £4 > if ||q|| is sufficiently large. 

Proof: In the following we show how to construct the 
function ^(q), /(q) and /i(q) based on /i(q). Since we only 
need to ensure that |/xi(q) — Mi(q)| < £4 for large |[q||, it 
is sufficient to construct the functions on the domain where 
||q|| > B for sufficiently large B. The function V and / on 
the domain ||q|| < B can be defined as any positive, bounded, 
continuously differentiable function, which is continuous on 
the boundary ||q|| = B. 




Fig. 3. Orthogonal grid (iiTegulai') in AI = 3-dimension. Tlie line integral 
between two points on the grid line (e.g. along the two paths marked by dashed 



line) depends only on the start point q 
of the chosen pathes 



and end point q*. It is independent 



In the domain of ||q|| > we at first construct a orthogonal 
grid such that each cell in the grid is a rectangle (see Figl3] 
for an example in M = 3-dimension). Start by a point q° = 
Q S M.¥ , the next cell in the dimensions i, j (see FiglUi has 



the grid points 

q^ = [Qi, Q, + AQ„ Qj, QMf, 
q" = [Ql,...,Q^,...,QJ + AQj,...,QMf, 
q"^ = [Qi, Q^ + AQ,, Qj + AQ,, Quf- 

The length of the cell AQi , AQj is determined by the equation 



^J.i{...,q,,QJ,...) +AQj,...)dqi 



AQj 

flj{...,Qi,qj, ...) - flji...,Qi + AQ„qj, ...)dqj. 

(42) 

Condition 2) in Theorem [T] implies that in the region ||q|| > 
B for some large constant B, the function p,i{q) decreases 
with increasing qj and flj{q) decreases with increasing qi as 
well. Hence 



fj.^{...,qi,Qj, ...) - .,qi,Qj + AQj, 
flj{...,Qi,qj,...) - flj{...,Qi + AQi,qj, 



.)>0 
,.)>0 



and Eqn.(l42t has positive general solutions with AQi, AQj > 
0. Iteratively take q^, q""' and q'^ as start point, we can extend 
the grid until it covers the subdomain in the dimensions i, 
j. Based on the existing grid lines in the dimensions i, j 
(e.g. the line q'^q^ in Fig. [3]), we can repeat the process in 
a further dimension k and construct the grid in this dimension 
(the grid q°-q''-q'^-q-^). Since relationship of AQi and AQj is 
determined by the definition of fLi (q) on the particular points, 
each rectangle in the grid has different height and width so 
that the constructed grid has a irregular pattern. 

Denote the path starts at via q^ to as Sabd and the 
path starts at via q'^ to q"^ as Sabd^ Eqn.(l42b ensures that 
the integral of the function /i(q) along the path Sabd equals 
the integral along the path Sacd^ which is 



/ M(q) ■ ds 



AQ, 



^j.i{...,q^,Qj,...)dqi 



AQj 



Hj{-.-,Qi + AQi,qj,...)dq.j 





AQj 



fij{...,Qi,qj,...)dqj 



AQ. 



^ii{...,qt,Qj + AQj, ...)dqi 
/ fi{q) ■ ds. 

■J Snnrl 



(43) 



Since Eqn.(l43]l holds for all cells of the grid, the integral 
between arbitrary two grid points along any grid line has the 
same value. Hence the vector field /i(q) can be considered 
as "path-independent" along the grid lines. Then we define a 



function /(q) whose value on the grid line as the integral of 
/i(q) along the grid lines, i.e. 

/(q*):-/(Q")+ / K<D-ds, 

Js 

where q* is a point on the grid line and S is an arbitrary path 
between q* and the initial point Q° along the grid lines. 

Define a new vector field by v{q) /(q)M(Q)' the Une 
integral of i>'(q) along the path Sabc is 

/ iv(q) • ds= / /(q)A(q) • ds 

Sabd Sabd 

/(q)rf/(q) 



j iv(q) • ds. 



Thus the integral of the vector field i'(q) between two grid 
points along the grid lines is also independent of the chosen 
paths. Then we define a scalar field ^(q) whose value on the 
grid line is given by 

V{cC) :=1/(Q°)+ / /(q)A(q)-ds. 

Js 

The value of /(Q°) and V{Q°) at the initial point Q° can 
be choose as an arbitrary positive constant. Since /ij(q) > 0, 
Vi e M, we have /(q*) +00 and V{(\*) +00 as 
||q*|| ->+oo. 

Once the value of V{c^ ) is fixed on the grid lines, we obtain 
the value of V inside a grid cell by the linear interpolation of 
y(q*) along the lines parallel to the diagonal line (see Fig|4]i, 
i.e. in the lower triangle with + < 1, is defined 

as 

V{...,Q, + ^q,,Q,+^qj,...)^K,V{ci') + KjV{ciJ), (44) 
where 



AQjAg, + AQ.Ag, ' 



q^ = [Qi,...,Q, + Agi 
q'^ = [Qi, ...,Q^,Qj + Aq-j 



AQ, 
AQ, 



Aqj,Qj, ...,QmV, 



■Aq„...,QMY 



and in the higher triangle with + > 1, y is defined 



as 



where 



AQj AQ, - AQj Ag, 



2AQ,AQj - AQjAq, - AQiAqj 

AQjAQ, ~ AQ.Aqj 
2AQ,AQj - AQjAq, - AQiAq, 



q^ ^ [...,Q, + Aq, 



AQ, 



AQ, 

^[...,Q^ + AQ,,Qj + Aq 



Aqj -AQ,,Qj + AQj 



Eqn. (l44l i and (l45l l determine the value of T^(q) on the 
orthogonal planes stretched by the grid, then the value of 
V^(q) in the space between these planes is calculated by the 
linear interpolation of the existing value in further dimensions. 
Similarly, we can also define the value of /(q) in the entire 
domain. 




y(...,0, + A9„0,+Agj,...) = K,Viq') + KjViq''), (45) 



Fig. 4. The value of y(q) := Qi + Agi, Q, + Agj, ...) is calculated 

by the linear interpolation between the value V(q ) and y(q^) defined on 
the grid Hues. The Hne q^q-' is parallel to the diagonal q''q'^ 

Observing the function V^(q), we can see that it is continu- 
ous in differentiable in each subspace bounded by the 
grid lines and diagonal lines. For two points q and q' which 
he in the same cell, under Condition 1) in Theorem[T]we have 
(q) - P-i (q')l < ei and hence \ f (q) - / (q')| < £1/ (q) 
for arbitrarily small ei > 0. Then for Eqn.(l44li it holds 

AQ, 
AQ, 

AQ, 
AQ, 

and further 

y(q) ^^(q'') + / (q) (fl, (q) + £,(q)) Aq, 

+ /(q) (Mj(q) + fj(q))Ag„ 

where the deviation ei{q), £j{q) — > as ||q|| +00. 
Similarly we can also obtain the same result for Eqn.(|45]l. 



Viq') = V{qn + f (q) ifM (q) + £«(q)) [Aq.. + ^Ag, ) , 
F(q^) - ^(q'') + / (q) (/I, (q) + e,(q)) ( Aq, + ^Aq, 



V{q} 




Fig. 5. The Lyapunov function V{q) is differentiable inside the subdomain 
between q".q'',q'^ and the subdomain between q'',q'^,q'^ 



Then the partial derivative of V is 



dqi 



f (q) ifL, (q) + £4) . 



for arbitrarily small £4 > and we obtain the Lemma |2] ■ 
It can be shown that /(q) and V^(q) constructed in Lemma 
|2]are positive and grow to infinity as |jq|| +00. Now we 
use the function V"(q) and /(q) in Lemma |2] as the Lyapunov 
function and the stability measure function respectively. It can 
also be shown that Al/(q") is bounded if q" lies in some 
compacted region B and the arrival rates af and transmission 
rates r" are bounded. Hence the Lyapunov condition ( |23] | is 
satisfied. 
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Fig. 6. The drift Ay crosses 5 subdomains, which can be written as the 
sum of the difference between y(q"), V{q_W)^ and V{q"+^) 



Next we consider the drift Ay(q") in Lyapunov condition 
(l24l i where q" ^ B. The connection between q" and q"+^ 



probably pass through multiple differentiable subspaces of 
V{q) (see Fig|6]l, so we denote the intersection of the connect- 
ing line and the boundary of the subspaces as q 



(1) 



the difference as Aq'^^ 
The drift is written as: 



,(1) 



q",..., Aq 



(0 



q*^^ and 
q(i+i)_q(0. 



AF(q") 

=E |T/(q"+i) - F(q(^)) + J2 ^(q'^') - "^(q^'^) 



1=2 



+ F(q('^) 
f L+l 



F(q")|q"j 

=E<' ^/(qW)/i(q(')).AqW 

I 1=1 
C L+l 



1=1 



Aq(' 



/(q(')) 



Where q''^ is some point in the l-th subspace. Since the 
arrival rates af and the transmission rates r" are bounded 
for all i G M, the difference ||Aq|| = ||q"+i - q"|| is 
bounded. Thus according to Condition 1) in Theorem [T] we 
have |/i,(qW)-/i,(q(i))| < ei and |/(q(')) - ./(q(i))| < 
ei/ (q'^') for arbitrary ei > if ||q'^'|| is large. The drift 

Al^(q") 

f L + l 

<E ./ (q(i))A(q^')) • E ^q^'^ + ^3/(q" 
I 1=1 

<E{/(q(l))/i(q(l)) . (q"+^ -q")|q"} +a3/(q" 

where 0-3 is some small constant. 

Using the previous result in ( |39] l, it holds 

AF(q") 



L i=i 



q" +^3/(q") 



<-0/(q")+a3/(q") 

< - ^7(q") 

for some 6*' > if ||q"|| > B, for some B > 0. The drift is 
negative thus the Markov chain is positive recurrent. 

At last, we prove that the chain is also f-stable for the 
magnitude function /(q). We can write 

E{F(q"+l)|q"} 

<E { F(q"+i)| q" > B] Pr (q" > B) 

+ E{ V^(q"+i)| q" < B}Pr(q" < B) 

<E{l/(q")-6l7(q")|q" > B} Pr (q" > B) 

+ E { V{c{'+^)\ q" < B) Pr (q" < B) 

<E{V^(q")}-0'/(q«)+C5, 

where C5 is some constant satisfying 

C5 > E { F(q"+i) I q" < B] Pr (q" < B) 

+ E { 0'f{cC)\ q" < B] Pr (q" < B) . 



Using the telescoping machinery, the summation of the drift 
over N time slots yields 



N 



, {Viq^)} < E {F(ql)} - 6' ^ E{/(q")} + N ■ C,. 



n = l 



since V{q) is non-negative function, it holds 



N 



< 



E{V{q^)} N-C, 



n=l 

Hence we have 



limsuplVE{/(q«)}<Mi:(^ + ^ 



< +00 



which completes the proof. 

B. Proof of Theorem \3\ 

Considering the first case in Theorem[3] we define the set of 
time slots in which the change of fli (q(«)) is not negligible 



n : n (q' 



n+l\ 



A(q")|| >e 



for some constant e > 0, where the superscript is again used 
to denote the index of the time slot. Suppose there is some 
constant 7 with < 7 < 1 and 

1 ^ 

-Y.^{neAfv}>J + e{N), 

n=l 

where e{N) — > as — » +00. If the difference be- 
tween p. (q""*"^) and p, (q") is larger than e, then the ex- 
pected rate allocation f| -.^ E{r^(h",q")} and f^+^ 
E{r'''(h"+^, q"+^)}, which are determined by p, (q"^^) and 
p (q"), also have non-negligible difference. Note that this 
assumption is valid only if the normal vector p is unique 
on every boundary point of C{P). Generally two different 
normal vector p and p' might lead to the same boundary 
point of C{P), where the boundary has no unique supporting 
hyperplane. Fortunately, to disprove the throughput-optimality 
we only need to consider certain rate region C{P) whose 
boundary is differentiable everywhere and the normal vector 
is always unique. Since both of and f^^^ lie inside C{P), 
for a boundary point r* of C{P) and the corresponding normal 
vector fi* we have 

=fi ■ arg max fi (q ) • r 

r'GC(P) 

+ fi*^ ■ aig max /i(q"+^)^-r' 

r'eC(P) 

<2fj,*^ r* -e{e), 

where 9{e) is determined by the difference e between 
/i (q"+^) and /i (q") with d{e) > 0. Considering the queue 



states on some even time slots N = 2,4, it holds 
/x*^E{q^} 

N/2 

=fi*^ J2 a" - r^(h2" , q2«) + 

n=l ^ 

-r^(h2"+i,q2«+i) 

N/2 

= Y: (2m*^P* ~ M*^ (f|" + i'r') I {2- ^ ^fv} 

n = l 

-f,*^ (f|"+f|"+i)l{2neAAy}) 
>^ (2/x*^p* - 2/x*^r* + (7 + siN)) 0(e)) (46) 

Suppose the expected arrival rate p* is close to the boundary 
point r* so that 

fi*^r* - fi*^p* < 6' 
for some 6' > 0. Combining with ( |46l ). it holds 

M*^E {q(iV)} > ^ (70(e) - 29' + e{N)6{e)) 
Since e{N) ^ as ^ +00, if 9' < ^9{e), we have 
lim fi*'^E{q{N)} = +00 

N^+oc 

and the Markov chain is not strongly stable. Suppose the 
variance a{n) and h(ri) is sufficiently small, so that for some 
constant Ca > the probability 

Pr{p*^q{N) < Ca} 

decreases sufficiently fast when ^'(iV) := /i*'^E {q(A^)} 
increases, i.e. 3Cb,K > with 

^^(i^pr ^ {A**^q(^) <Ca}, V*(iV) > Cb, 

for some constant F > 0. Define Nc := min{iV e N : 
'l'(iV) > Cb}, the expected occupation time of the set 
^ := {q : p*^q < Ca} is given by 



00 . 

< iv, + y K ^ 



N=N^ 



<If(iV)i+r 



1 



N=N^ \{{10^O')N)'+^J 

< +00 

and the Markov chain is transient. 

For the second case in Theorem [3] we choose the expected 
arrival rate vector p* close to the j-th corner of the ergodic 
achievable rate region with 

r*-p*<9'. 



for the user ?' 7^ i and some constant 9' > 0, where r* := 
maxjgi^jpj fj. According to ( (15b we have for some constant 
e > and 7 with < 7 < 1 

1 ^ 

-^I{/i,(q(n))<l-e}>7 + e(iV), 

n— 1 

where e{N) as iV ^ +00. It implies that for some 
9{e) > 0, it holds 

1 w 

-^l{r*-r;(n)>0(e)} >7 + £(A^), 

where r^- (n) := E{r''(h", q")} and the corner point 
[0, r*, ...] can not be achieved. Then we have 

N 

n=l 
N 

n=l 

-r'^{n)l{r* -r'^{n)<9{e)}) 
>N {j9{e) -9' + e{N)9{e)) . (47) 

Choose 9' < ^9{e), we have 

lim E|gf|=+c» 

and the Markov chain is not strongly stable. Similarly we can 
also show that the Markov chain is transient if the variances 
of a(n) and h(n) are sufficiently small. 
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